Localization and tracking of an item in a real environment is an interesting part of many useful endeavors. Depending upon the endeavor, the tracked item may be virtually anything, but routinely involves a person, a camera, an electronic device or a product. Some of the useful endeavors relate to navigation, item tracking, object recognition, 3D reconstruction, virtual reality and augmented reality. These endeavors may be embodied in hardware or software or both.
In one particular example, computer vision and/or optical analysis mechanisms may be used to determine a pose (e.g. position and orientation) of the camera relative to the real environment or a camera ego-motion (e.g. the 3D motion of a camera within an environment). In more particularity, computer vision techniques often involve analysis of images of the real environment being used to deduce pose or other position or localization information. The images may be obtained from any source, such as directly from one or more cameras/sensors or by acquiring image sets that were captured earlier. In either case, the images may be stored for analysis and the images and metadata associated with the images may be retained in a database.
Various computer vision methods may be used to estimate or determine tracking or localization information, such as information embodied in a camera pose estimation. For example, computer vision concepts, such as simultaneously localization and mapping (SLAM) and key-frame tracking, may be employed. In some SLAM implementations, frame to frame tracking matches image features of a current image with image features of previously captured images for simultaneous localization and 3D reconstruction. As another example, global tracking, such as key-frame tracking, matches image features of a current image with a reference image feature for online tracking. The reference image features may be obtained from a pre-constructed 3D model of the real environment (i.e. a cloud of 3D points of reference features) that is built offline.
One way to improve on computer vision-based localization and tracking techniques is to add environmental information to the analysis. For example, environmental factors, such as gravity or other known environmental situations, may be employed. As another example, where cameras have inertial measurement units (IMUs), environmental information may be obtained therefrom and associated with the images to improve localization and tracking. In addition, wireless signals, such as radio and ultrasound, may also be used to aid localization and tracking. For example, location fingerprinting helps locate a device by using WiFi, Bluetooth, or a combination of device sensors.